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ABSTRACT _ • 

The National ecroncil on Measurement in Education 
(HCHEI members • expectations cf teacher and student ^attitudes toward 
standardized testing are N compared with the actual attitudes of these 
arou'Fs tc ascertain whether the perceptions of test specialists are 
similar tc those of test users. Of a. random sample of SettE members^ 
assicneS systematically to three grade ranges , 59 percent responded 
to'.attitudinal questionnaires as they thought teachers and students 
would in grades in grades 5-8, or in grades 9-12- A comparison 

of t£e HCME responses is made with -actual responses _ pre viqusly 
obtained from 3,3C|6 teachers and 71,763 students (90S response, ratej 
in grades 12 who answered identical questionnaires immediately 
subsequent tc an administration of a standardized achievement test* _ 
Highliohts cf results indicate:^ (1) teachers are actually more 
knowledgeable abbut^ and supportive of standardized achievement tests 
then NCME members expect them to be: (2) grades K-4 students^ 

attitudes are far more positive than expected; (3) grades 5-12 

students* attitudes reflect that. the tests are less difficult, less 
unfair, and less likely to' stimulate stress than expected. Attitudes 
should be monitored in order to assist non-specialist test users 
successfully* (EL) 



:* Beproductidns supplied by EDES ar^ the best thatocan be made- * 
* from the original document* * 



Abstract 



U S -D€ PART ME N t. OP H E.AL t H. 

* E P U CAT 10 N_* WELFARE 
NATIONAL INSTITUTE OF 

• EDUCATION 

THIS DOCUMENT HAS BEEN REPRO- 

OUCE-O-EXaCJ-Ly: REcEivE&.F.ROM 
T h £ > £ R SON .0 R O R C a NiZA J)0 N J0.RJG.I_N? 
ATING IT POINTS op VIEW OR OPINIONS 
STATED DO N"t NECESSARiijy-REP-RE-- 
SE.N.T 0FF!£L'AL..NATi.ONAL !N$TlTjjT£ OF 
E DuC A T 'ON POSITION OR POUCY 



LU 



Standard .*d Testing as Viewed by Test Specialists and Users 



Michael D. Reck and Frank^P. Stetz 
Psychological Corporation 
; *ew York, NewTork 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



T 0 THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)/* 



as 



NCME members 1 expectations of readier and student attitudes toward 
standardized testing ware compared with the actual attitudes of 
these groups. A random sarnie of NCME members completed the same 
questionnaire as was usee .n a previous nationwide study of teacher 
and student attitudes. NCME members were asked to respond as they 
believed teachers and students would. Results indicate significant 
discrepancies between actual teacher and student reactions and the 
attitudes expected of them by NCME respondents. Teachers and students 
' reacted far more positively toward such instruments than NCME members 
expected. 
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Introduction* • 

At last year's NCME and ^ERA annual meetings, Stetz and Beck (1979) and 
Beck and Stetz (1979) pr • -erted- findings of a nationwide survey of pub- ' } j 
lie and nonpublic teacher m£ .students In Grades K-12. Their inquiry, 
immediately subsequent to an administration- of a standardized achieve- 
ment test*, dealt with, such variables as: teachers' opinions and personal 
feelings toward standardized achievement tests, their opinions of the . 
amount of standardized testing in their school' systems, -their personal 
use of such test results, and their predilection toward various test- 
related policies. In addition, students were asked such questions as _ 
"How many of you were nervous dust before you took this test?", "How dp 
you feel' about the test now?", and "Would you like to ta|e a test like 
this one next year?" 



Their results showed, among other things, that standardized testing is 
viewed with some skepticism and anxiety. In general, however, test re^ 
suits are considered useful for certain purposes and cause less anxiety 
in students than teacher-made tests. 

^ ' i 

Objective 

With the above in mind, a replication of the above study, was conducted 
with a random sample of NCME members to ascertain whether the percep- 
tions of test specialists are similar to those of test users. This 
^question has been raised by Various NCME members 4ri the past (e.g., 
Olejnik, 1979; Stetz S^eeV 1978}. 
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The same teacher and student questionnaires used in the 1979 Stetz and 
Beck studies were used. The only modifications were 1) a reminder to 
HEME members at the- top and bottom of each page to respond 'as they 
thought a randbS sample of teaehers (or students) would, and 2) a re- 
quest to respond in .pereents to eertain student questions rather than 
' as^for hand counts of students as was done in the original study. 

* 

Among other questions, the teacher questionnaire contained 11 semantic 
, differential scales surveying NCME members* impressions of teachers' 
general opinions and feelings. Examples of the bipolar adjective de- 
scriptors included: hel pful -harmful , unbiased-Biased^calm-anxious, 
supportive-antagonistic. ; The student portion was originally administered 
orally by the teacher; teachers recorded the number of hands, raised in 
-response to the questions. In this survey, NCME members were asked to 
respond as they thought students would. 

Method 

A random sample of 500- NCME members was selected from the NCME membership 
list. The' teacher and student questionnaires were segmented into three 
groups; in the questionnaire cover sheet, NCME members were asked to re- 
spond as .they thought teachers and students would either in Grades K-4, in 

or in Grades 9-12. Assignment to grade range was done systematically. 

* * » 

Materials sent to each NCME member sampled included a cover letter, the 
questionnaire* arid a. postpaid return envelope. Approximately two 
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weeks later, a followup letter Was se^it to each sample member, urging 
their cooperation. 

To control for response bias due to knowledge 'if the results of the 
original teacher and student study, the questionnaire was may led well in 
advance of the 1979 NCME/AERA annual meeting. The deadline date was 
also prior td the annual meetings. 

Results r 

- . _ ' \ : 

Table 1 summarizes the number of respondents to the two surveys. The 

student and teacher counts 'represent a r&turn rate of over 90% of those 

sampled; the response rate for NCME members was 59% of those contacted. 

One of the questionnaire items asked about the amount of standardized 
testing that takes place in the teachers 1 own school system. Table 2 
summarizes the results. Respondents indieated- Jl to6 great/ "about 
right," or "too little." N6ME respondents thought many more teachers 
than was actually the case would feel there yas too much testing in, 
their system. Almost 60 percent of the NCME members thought teachers 
would reply "too much," while less than 20 percent actually did so. 

A further question concerned teachers' personal use of standardized 
achievement test results. Respondents indicated whether they made 
"little,"' "some," or "considerable" use of such data. As opposed to 
the above results, the perceptions of NCME members were close to the 
teachers' ratirfgs eh this question. Slightly over half of each group' 
indicated that "some" use was made, while just under 10 percent of 
both groups indicated "considerable" teacher use. 
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Table 1 

Total Number of NCME Members , Teachers and Students Responding by Grade 



Grades 


NCME Members 


Teachers 


Students 


K-4 


103 


1516 


31 ,956 


5-8 


95 


1210 


28,246 


- 9-12 


. 86 


580 


11,561 


Total 


254 


3306 


71 ,763 



Jabia 2 • 

Gompansbhs Between NCME Members* Impressions and 
Teachers' Opinions of the Amount of standardized Testing 
in the Teachers* School Systems 



Amount of Testing 
in Your System: 


NCME Members' 
of Teachers' 


Impressions 
Opinions . - 


Teachers' 


Opinions 


K-S • 


9-12 


£-8 


| 9-12 . 


Too Great 
About Right 


60 


57 


26 


16' 


39 


38 


70 


64 


Too tattle 


1 


. 5 


6 


13, 


No Answer 


0 


;0 


4 


7 










* 



'Numbers expressed in percents. r 

7 



Table 3 provides a comparison of teachers' personal feelings toward > 
standardized achievement tes* c with the NCME members 1 expectation of 
teachers' feelings. Teachers are actually more interested in, calm, 
comfortable, and knowledgeable about, ,ahd supportive of standardized 
achievement tests^than NCME respondents expected them to be. For 
most items, NCME members' impressions of teachers' attitudes were more 
/vegative than reality. 

Another set of "semantic differential scales provided additional com- 
parisons of opinions concerning such tests. Teachers generally con- 
sider such tests more helpful* fair * and useful* less biased, but 
harder than NCME members expected them to. } > 

Table 4 indicates the two groups' responses to four questions concern- 
ing moratoriums oh various tests. Differences between the groups here 
are inconsistent. Overall, NCME members expected more teachers to 
favor moratoriums than was actually found. However, the differences 
between groups were small for reactions to moratoriums on aXL stan- 
dardized tests and on standardized achievements tests. In contrast, 
about half as many teachers as expected by NCME respondents favor 
moratoriums on standardized intelligence and state-mandated tests. 
NCME members appear to consider teachers far more negative toward 
thlse two types of instruments than is actually the case. 

Teachers 1 opinions concerning various other test-related practices are 
summarized in Table 5. NCME members 1 expectations and teachers' 
.attitudes were somewhat discrepant in the use of tests for accountability 
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Table 3 



Comparisons Between NCME Members'. Impressions and -j 
Teachers' Personal Feelings Toward Standardized Achievement Tests 
; (Grades K-12) 



Dimensions 



NCME Members '. impressions 
of Teachers' Feelings 



Teachers' Feelings 



Calm 

1-2 

3-5 

6-7 
Anxious 



10 
73 
17 



42 
51 

7 



Comfortable 

1-2 

3-5 
•6-7 

Uncomfortable 



6 
80 
14 



•38 
55 
7 



Interested 
1-2 
3-5 
6-7 

Uninterested 



16 
76 
8 



37 
55 

8 



Knowledgeable 
1-2 
3-5 
6-7 . 

Not Know! edgeab - ! e 




34 
64 
2 



-Supportive 
1-2 
3-5 
6-7 

Antagonistic 



3 

83 
9 



32 
63 
5 



1 



Numbers are expressed in pereents. 

2 Although the questionnaire used a seven-point scale* data were 
collapsed into a three-point scale range for- summary purposes. 
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Table 4 

Comparisons Between NCME Members' Impressions and ■« 
Teachers Favoring Various Moratoriums bri Standardized Tests 

(Grades K-12) 



Percent of Teachers Personally 
Favoring a Moratorium on: 


NCME Members' Impressions 
of Teachers Favoring 


Teachers Favoring 


All Standardized Tests 


19 . ; 


16 


Standardized Intelligence Te£ts 


/ 

58 


26 


Standardized Achievensent Tests 


29 


19- 


4 

State-Mandated Achievement 
Tests 


62 


31 



1 



Numbers expressed in percents- 



Table 5 

Comparisons Between NCME Members- Impressions and 
Teachers Favoring Various Test-Related Policies 

(Grades K-12) 



Percent of Teachers Personally 
Favoring: 


8GME Members* 
Impressions of Teachers 
Favori hg 


Teachers Favoring 


The use of "competency" test 
results to determine high school 
graduation 


• 42 


62 


Additional training of school 
personnel in test interpretation 
and use 


* 77 


61 


Increased use of test results 
for school "accountability" pur- 
poses 


12 


- - 

26 


Increased use of criterion- 
referenced tests 

i 


80 


38 


^uftfcers expressed in bercent*, 

■m 1 


• 9 
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and additional draining of school personnel in test interpretation arid 
use. Their views were not greatly discrepant {42% vs. B2%) for using 
competency tests to determine high school graduation. Finally, more 
than twice as many MCME members expected teachers to favor increased 
eriteribri-referericed test- use than was true. 

The questionnaire listed a variety of purposes for which standardized 

_ . . . ... _« _ _ 

achievement tests are used. Respondents indicated whether such tests w 

were useful for these purposes, fable 6 sumnarizes the results. Arbitrarily 
using a difference of 10? as indicating "significance," the results 
indicate NCME members overestimate teaehers 1 dispositions toward test use 
for reporting to newspapers, boards of education, arid parents; for 
screening -for special education; and for comparing students and systems 
with national reference groups. On the other hand, teachers reaet more 
positively then expected by NCME members to using test data for measur- 
ing "growth, " planning instruction; for individuals or classes, evaluat- 
ing teaching methods arid material s* arid evaluating teacher performance. 

Table 7 compares Grades K-4 students 1 - jmpr es s i o ns of standardized 
achievement tests with the reactions expected of them by NCME members. 
Overall, students' attitudes were far more positive— both before and 
after testing— than expected by NCME members. 

Finally, Table 8 compares Grades 5-12 students 1 impressions of stan- 
dardized achievement and ( teacher-made tests with the reactions expected 
of them by NCME members. NCME members greatly overestimate the diffi- 
culty of standardized tests and underestimate the difficulty of teacher- : 
made tests. NCME members also overestimate students* negative pereep- ; 



Table 6 



Comparisons Between NCME Members 1 Impressions and 
Teachers Who Consider Standardized Achievement Test 
Results Useful for Various Purposes 

(Grades K-12) { 



Standardized Test Results are 
Useful to: - - 


NCME Member Impressions 
of Teachers 1 Opinions 


Teachers' Opinions 


Report tc newspapers 


33 


VQ 


Report ±o boards ot education 




52 


Report to parents * 


82 


-J- -67- < 


Rpnnrt nroorp?^ to stndpnts 

ixcpv t — l» — yf* W*jl vJ J WW — w ^mjlttlt WJ 


59 


56 ^ 


Measure educational status of 
individuals 




fil 


Measure educational "growth" 
of individuals 


56 


77 


Screen special education 


73 


55 


Help plan instruction for 

I nul VI QUa 1 5 




V 


H&ln nlan instruction for 

HC 1 p U lull II Iw U UV« b iwll 1 w 1 

class qroups : 


47 


65 V 


tictcuw sysoeni^wjde general 
strengths/weaknesses 


81 


75 


Help evaluate teaching pro- 
cedures or methods 


15 


* 

34 


Help evaluate instructional 
— . • materials 


23 


41 


Help evaluate teacher per- 
formance . 


5 


21 


Compare students with 
national peer groups 


91 


58 


Compare classes in a school 


36 


,30 


Compare schools within a* 


41 


36 


• Compare a system with systems 
across the country 


-75 


56 
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Comparisons Between N^E Members^ impressions and 
Students' Opinions Tpward an Administration of 
- a . Standardized Achievement test 
(Grades k-4) 

t • i 



Dimensions- 



FEELINGS TOWARD 
TEST: 








PERCENT NERVOUS " 
1 JUST BEFORE TEST 



WOULD YOU WANT TO 
TAKE ANOTHER SUCH 
TEST NEXT YEAR? 

PERCENT ANSWERING:" 
YES 



NCHE Members 1 Impress i ons 
of Students' Opinions 



Before Test After -Test 



12 



48^ 



23 



28 



33 • 



4t 



16 



11 



67 



40^ 



NO 



DON'T KNOW/NOT SURE 



52 



18" 



Student Opinions 



•2 



Before 



After 



37 



58 



14 



22 



10 



Ta- 



il 



56 



38 



48 



14" 



1 



The first percent represents NCME members' impressions of students' 
opinions just befdre th? test administration; the second percent 
represents NCME members 1 impressions of student opinions just after 
the test administration.. ' ' - 



Theflrst percent represents students' feelings Just before the test 
administration; the second percent represents students ' feel ings 
just after the test administration. 
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* - Table 8 ■ 

CooparisoBs. Be'bireefl NCME M^bersA_JiSSssjons_ahd V , 
Students' Opinions -Toward an Administration of 
a Standardized Achievement Test 
(Grades 5-12) : 
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Questions 


NCME Members 1 Impressions 
of Student Opinions* 


Students- Opinions — 


Sr. 5-8 - Sr. 9-12 


fir. 5-8 fir. 


"fi§1ch_ type of tet_do_you _ _ 
think 1s usually harder: type 
of test you just took or the 
type of test your teachers * 
make up?" 

-Standardised Tests^ » 


1 

60 


— \ ■ 

r 

M 

65 - 


X 

36 


X 

/ 

16 , 


-Teacher-made tests 


32 


26 : 


54— 


— 73 


-Don't know >^ 


8 




10 


11 


"The test you just took Is sup- 
posed to sbow_wtoat_youa1 ready 
have learned and what/you do 
hot know yet. So-yoii think the 
test questions were generally 

ToTTf 

-Yes' 


56 


49 


74 


78 


-No 


33 


38 


16 


U 


♦ -Don't Know 


It 


13 


10 


n t 


"How do you feel you did on 
this; test?" 

-Hell 


l *t 

17 


16 


25 


31 


-Ok/All Rlqht - 


— 49 — 


*rD 


AO 

— **o — 


AQ 

**o — 


-Poorly 


Zu 


C 1 


O 
o 


C 

D * 


— -Don't Know — — — 


— i** - — 


18 


19 


16 


in ycncra i » now oo you • ctr i 

about t^e type of test you 
^j^st^ookr 

-Pos^tlve* 


- 39 


36 v 


29— 


• 18 


t ^Tfe^at1ve— 


27 


36 


27 


29 


-Neutral/Unsure I 


—33— 


—28 . 


• 44 




"Would you be interested in 
r i no i ng out your_ scores on tne 
test you just took?" 

-Yes — - — * — 


— — ^i— 


—76- 


9C 


84 


—Ho 


16 


— «— 


—6— 


i 10 


-Not Sure 


8 


9 


4 


6 


"Would _^u_11ke^e_^ance_tb 
discuss with your teacher the 
correct answers to the tet 
you just took?" 

-Yes 


67 


60 


48 


31 




22 


26 


39 


55 


-Not Sure 


11 


13 


13 


1* 


"How many of you get nervous 
just before you take a test 
like the one you just took?" 


67 - 


» 

62 


'37 ■ 


. V 

13 


"How many of you get nervous 
just before you take the kind 
of test your teachers make moV* — 


59 




61 


71 ^ 
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tibris of the fairness of the questions on standardized achievement tests; 
In general i NCME "members ' views and students 1 feelings on how students 

. did on the test arid students 1 general perceptions of such tests were 
similar. NCME members overestimated the nQrab^r of students who would 
like the chance to discuss the correct answers to the test. Finally, 
NCME members overestimated the number of students who get nervous prior 

, to taking standardized' achievement tests. 

Edu cational -Significance 

NCME is a professional association of measurement specialists that is, 

by design* practitioner-oriented. As such, it is important that its 

members be sensitive to the attitudes of test users toward assessment 

■ * • 

devices- This study highlights areas in which perceptions of specialists 
about user attitudes vary markedly from reality. It is important to 
monitor the relationship between perceived and actual attitudes if NCME 
is'to be successful in assisting riori-specialist test users in making 
- better use of educational assessment. This study provides data for such 
a process and may help to direct the organization's resources toward 
shrinking the gap between perception and reality. . 
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